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In bilateral accounting of resource consumption both the consumer and provider independently mea- 
sure the amount of resources consumed by the consumer The problem here is that potential dis- 
parities between the provider's and consumer's accountings, might lead to conflicts between the two 
parties that need to be resolved. We argue that with the proper mechanisms available, most of these 
conflicts can be solved online, as opposite to in court resolution; the design of such mechanisms is 
still a research topic; to help cover the gap, in this paper we propose a peer-to-peer protocol for 
online dispute resolution over storage consumption. The protocol is peer-to-peer and takes into con- 
sideration the possible causes (e.g, transmission delays, unsynchronized metric collectors, etc.) of 
the disparity between the provider's and consumer's accountings to make, if possible, the two results 
converge. 

1 Introduction 

The general scenario of our research interest is the consumption of computing resources (storage, band- 
width, computation, etc.) offered by providers to remote users (consumers) over the Internet. The 
consumer regards the resources as a service reachable through a user's interface and pays for it on a pay- 
per-use basis. Central to this scenario is resource consumption accounting. Currently, most providers use 
unilateral provider-side accounting based on metrics collected by devices deployed within the providers' 
premises. An alternative and innovative approach is bilateral accounting where both the consumer and 
provider independently measure resource consumption and verify the parity of the accounting results Q. 
A potential problem here is the emergence of potential conflicts derived from divergences between the 
independently produced accounting results. The practicality of bilateral accounting depends on whether 
most conflicts can be solved online, as opposite to off-line resolution; this issue is still an open research 
question. To help cover the gap, in this paper we present an online peer-to-peer protocol for dispute res- 
olution over resource consumption. To meet space and time constraints and focus the discussion on an 
specific and practical example, we deal only with storage consumption and in particular we concentrate 
on a rather simple scenario where the consumer can only upload data to an incremental storage service. 

An abstract view of our scenario of study is shown in Figure [T] where Provider represents an storage 
service and Consumer represents the consumer of the service which can be a single individual or a large 
enterprise with scores of employees or a university with thousands of students. As shown in the figure, 
consumer and provider deploy their own resource accounting services {RASc and RASp, respectively) 
within theirs respective infrastructures. A RAS is composed of three components: a metering service 
(MS) responsible for collecting raw metering data about storage consumption; an accounting service 
(AS) that retrieves the metering data and applies an accounting model to produce accounting data; and 
a billing service (BS) that on the basis of the accounting data provided by the AS and pricing policies 
(e.g., discounts to golden customers, fines to late payments, etc.) produces the actual bill, say monthly. 
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Figure 1 : A bilateral storage accounting system. 



for the consumer. As shown in the figure, the consumer can access the service only through a storage 
interface — ^the service interface offered to consumers. 

The CCRP (Comparison and Conflict Resolution Protocol) is the central topic of this paper and 
represents the protocol that the consumer and provider execute when conflicts over storage consumption 
emerge, with the intention of solving them online. As shown in the figure, the protocol is executed by 
the accounting services when the difference between their independently produced accounting results is 
greater than an agreed upon value. We anticipate several sources of conflicts. For example, a primary 
source of conflics is the accounting model used by consumer and provider to compute accounting data. 
As some in the figure, to avoid this problem, in this paper we require that both consumer and provider use 
the same accounting model which is publish by the provider. Such a model is basically an algorithm that 
aggregates raw metering data (e.g., 300000 upload this week) and converts it into accounting records over 
agreed upon consumption intervals (10 Mbytes/Mon, 12Mbytes/Tue, etc.). Another source of potential 
conflicts is the techniques used by consumer and provider to collect metering data with their MSs. For 
example, the consumer might rely on interceptors whereas the provider — with unrestricted access to its 
infrastructure — ^might measure storage consumption directly from its file servers. At this stage of our 
research, and as shown in the figure, we assume that both consumer and provider use interceptors to 
collect metering data. There are others sources of potential conflicts, yet we will concentrate on the 
transmission time of requests and the accuracy of accounting intervals as these two parameters are the 
most relevant to the scenario of our interest. We will discuss the CCRP (as well as the interceptors) at 
large in the following sections. 



2 The Protocol 

The CCRP is an peer-to-peer conflict resolution protocol in the sense that it is executed between the two 
conflicting parties without the intervention of a third one, such as a referee or arbitrator. It is an online 
protocol in that it is executed immediately upon the detection of a conflict and as the service is deUvered 
to the consumer. More importantly, it is an evidence-based protocol in the sense that, on the basis of 
evidence provided by the conflicting parties, it tries to identify the source of the divergency between 



A. Mihoob & C. Molina-Jimenez 



5 



the two accounting results. It is worth mentioning that this approach departs from conflict resolution 
protocols based on Utility Theory which are interest-based in that, they take into consideration the 
conflicting parties' preferences and tradeoffs 1.10.1 . 

3 Assumptions 

We admit that the CCRP is still under development. In this paper we explore its feasibility in a very 
simple incremental storage consumption scenario described by the assumption discussed below. As 
explained in Section[8l we are planning to relax these assumptions in the future to generalise our scenario. 
We believe that the fundamental ideas (e.g. the architecture shown in Figure O discussed in the current 
scenario will still hold in a more general one. 

The scenario of study can be described by the following assumptions: 

1. The provider offers an incremental storage service where upload file is the only operation available 
to the consumer to alter its storage space and each execution results in the creation of a new file. 
This service is far from being a general scenario, yet it is still of some practical interest (e.g. in 
archival storage 121) and more importantly, it is good enough to explain our ideas. 

2. All the consumer's upload operations are requested from within its premises (see Figured]); con- 
sumers with laptops that roam outside the premises are not considered. 

3. The service is delivery continuously over an agreed period of time, for example, over a year. 

4. In the interest of accountability, the total period is divided into Consumption Intervals (CI) with 
SP and EP (Start Point and End Point, respectively) determined by the provider. 

5. Zero o more requests can issued by the consumer during the duration of each consumption interval. 

6. There are no gaps in the accounting line. Except for the last interval, the end of a given interval 
correspond to the start of the next one. 

7. The consumer and provider independently produce their accounting record about the storage con- 
sumed over each consumption interval. The two independently produced records do not necessar- 
ily match. 

8. The CCRP is executed for each consumption interval to compare the two independently produced 
accounting records and to try to solve potential conflicts. 

9. The provider's and consumer's clocks are synchronized. 

10. The interceptors used by the consumer and provider are deployed as shown in Figure[I]to intercept 
each consumer's request. 

1 1 . The MSc and MSp collect the following data about each request: Request id, Request Time Stamp 
(RTS) and Bytes Transferred per Request (BT). In addition, MSp also collects Request Received 
Time (RRT). 

4 Accounting Model 

The accounting model is used by both the consumer and provider to calculate the storage consumed by 
each request issued by the consumer. Under the assumption that the provider relies on conventional file 
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systems to implement his service, our accounting model considers the number of bytes uploaded by the 
request and the configuration parameters of the provider's file system. 

The number of bytes transferred by each request {BTreqi) is determined by the interceptors after 
intercepting and examining the request. 

The configuration parameters are inherent to the file system. In Our accounting model we consider 
the amount of metadata {MD) associated to each file and the size of the disk chunk (ChSize) — also called, 
size of disk cluster. Typical values of MD and ChSize are, respectively, 2KB and 4KB. In this order, the 
number of chunks consumed by a request can be calculated by equation [T] 



NofChi 



BTReqi+MD 
ChSize 



It follows that the storage consumed by a given request can be calculated by equation [2l 

SCUF = RoundUp{NofCh) * ChSize 



(1) 



(2) 



RoundUp represents a round up operation to the nearest integer and counts for the fact that disk 
chunks are allocated only in whole units. 

The amount of storage consumed within each consumption interval can be calculated as the sum of 
the storage consumed by each request issued within the interval; we represent it by equation |3] and show 
it graphically in Figure |2l 
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Figure 2: Storage consumed within each consumption interval. 



5 Two Potential Causes of Disparities over Storage Consumption 

In this section we will explain how mismatches between the consumer's and provider's accounting in- 
tervals can results in disparities between the consumer's an provider's accounting results. Likewise, we 
will discuss how transmission time impacts the accounting results produced by the two parties. 
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5.1 Consumption Interval 

The impact of potential mismatches between the consumer's and provider's consumptions intervals is 
shown in Figure[3] In the figure, CI and R stand, respectively, for for Consumption Interval and Requests. 
Similarly, SP and EP stand, respectively, for Start Point and End Point of a given interval. We refer with 
superscripts c and p, to consumer and provider, respectively. Subscripts represent the sequence number 
of the interval; for example, SP^ represents the start point of the consumer's interval number one. Notice 
that for simplicity, the figure assumes that the transmission time of the requests is zero. 
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Figure 3: Mismatch between consumer's and provider's consumption intervals. 

As suggested by the figure, it is quite possible that for a given interval the consumer's Start Point (SP) 
and End Point (EP) do not match the provider's. Such a mismatch is very likely to result in divergencies 
between the consumer's and provider's accounting records for the interval under question. In the figure, 
EP^ > EP[, consequently, Cl[ includes N requests more than the consumer's C/J'. Naturally, the length 
of the divergency between the two results depends on the amount of bytes transferred in each requests 
and more importantly, on the value of N >Q. As shown in the figure, length of the divergency for Ch 
depends on the value of A'^ and M > 0. 

In practice, this situation can arise when the provider does not offer precise information to the con- 
sumer about when to start and end a given consumption interval. For instance, most storage providers, 
like Amazon 13, and Nirvanix Q do not offer (e.g. in their Service Level Agreements) sound account- 
ing models to their customers. For example, in Amazon S3 service, the storage consumed by a given 
customer is calculated as follows: Amazon checks at least twice a day the consumer's storage space, 
it measures the amount of storage occupied by a consumer's buckets and multiplies the result by the 
amount of time elapsed since the last check. However, Amazon S3 does not state exactly when (in time 
units) they undertake their measurement. 
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As discussed in Section |4l the consumer and provider calculate storage consumption within a given 
consumption interval by equation [3] The requirement to make the two independently produced results 
converge is that both parties use exactly the same SP and EP for the interval under question. We anticipate 
two possible solutions to this problem. An alternative is to keep the consumer's and provider's metering 
services strictly synchronised; for example, the provider can notify the consumer when each consumption 
interval starts and ends. As shown in the pseudocode presented in Section 1631 in this paper we explore a 
second alternative where the parties exchange their SP and EP upon detection of conflicts between their 
accounting results. 

5.2 Transmission Time 

We define transmission time (TT) as the time it takes a consumer's request to travel from the consumer 
to the provider. In practical applications, TT is normally greater than zero, say of the order of 100 
milliseconds. In Figure IH TT represents the average transmission time. As shown graphically, this 
parameter can cause divergencies between the consumer's and provider's accounting results for a given 
consumption interval. For the sake of simplicity, let us assume that the consumer's and provider's SP and 
EP of a given interval are synchronised. Under this assumption, convergency between the consumer's 
and provider's accounting records can be achieved by compensating the provider's results by the amount 
of memory consumed by the requests in the wire, that is, requests issued in a given interval but received 
and counted in the following due to TT. 
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Figure 4: Impact of transmission time on storage accounting. 
Let us take an arbitray interval C/,. The consumer can calculate its storage consumption by equa- 
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tion[3l However, to compensate for TT, the provider would need to use equation |4l 

n 

SC = Y,SCUFi+\N-M\ (4) 

(=1 

where N is the amount of storage consumed by requests issued and counted by the consumer in 
interval C/,-i but received and counted by the provider in interval C/, due to the effect of TT, in the 
figure this time gap is shown as TTi. Similarly, M is the amount of storage consumed by requets issued 
and counted by the consumer in interval C/, but to be received and counted by the provider in interval 
C/,+1, due to TT; in the figure, this time gap is shown as TT2. Both N and M can be calculated by 
equation |3] Notice that for the first interval N is to be taken a.s N = 0. 

An equivalent alternative to compensate the provider's accounting results is for the provider (or the 
consumer) to shift its consumption interval to count for TT. This is the strategy taken in the pseudocode 
presented in Section [631 

It is worth keeping in mind that accounting records can be impacted simultaneously by both asyn- 
chrony of consumption interval and transmission time. The protocol presented in Section 16.31 handles 
the two potential sources of conflicts separately. First it tries to match the results by considering the 
asynchrony of the interval; if it fails, it takes TT into account. Failure to produce matching results leads 
to offline dispute resolution. 

6 The Model 

A crucial problem in accounting of storage consumption is the generation of non-repudiable evidence 
about the consumption. We address this issue with the help of the piece of middleware for Non- 
Repudiable (NR) information sharing presented in HO. The fundamental idea is that the middleware 
provides multi-party, non-repudiable agreement to updates to shared information which can be main- 
tained in a distributed manner with each party holding a copy. Essentially, one party proposes a new 
value for the state of some information and the other parties sharing the information subject the proposed 
value to application-specific validation. If all parties agree to the value, then the shared view of the 
information is updated accordingly. Otherwise, the shared view of the information remains in the state 
prior to proposal of the new value. 

The architecture of our solution is shown in Figure |5] NR Midleware represents the non-repudiable 
middleware. Similarly, RASc and RASp represent, respectively, the consumer's and provider's resource 
accounting systems. Non-Agreed NRData and Agreed NRData are files to store, respectively, non-agreed 
and agreed accounting records, as determined by the P2P online Dispute Resolution protocol. Records 
from the Non-Agreed log can be used in case of offline dispute resolution. 

The signed two-phase commit protocols works as follows: 

1 . The provider RASp calculates the accounting record SRf for a given consumption interval C/, and 
sends it to its NR Midleware which produces non-repudiation of its origin NRO{SRf) and sends 
NRO{SRf) and SRf to the consumer. 

2. The consumer's NR middleware validates SRf and NRO{SRf) and sends SRf to its RASc- 

3. RASc produces an accounting record SR'i for C/,, compares it with the SRf and produces a decision, 
decni. decni is essentially a binary Yes or No value. If decni = Yes, RASc sends decni to the 
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Figure 5: Architecture to support the onhne dispute resolution protocol. 

consumer NR middleware, otherwise, it triggers the online P2P dispute resolution. When this 
protocol is completed decni is sent to the consumer's NR middleware. 

4. The consumer's NR middleware sends the decision decni, non-repudiation of receipt of SRf, 
NRR{SRf) and non-repudiation of origin of the decision NRO{decni). 

5. The provider NR middleware validates decni, NRO{decni) and NRR{SRf). The protocol termi- 
nates with the provider sending non-repudiation of receipt of the validation decision to the con- 
sumer NRR{decni). 

6.1 The Provider's Resource Accounting Service 

We will elaborate on how the provider's resource accounting service work, with the help of Figure |6] 
which is an expansion of Figure |5] RASp is a service with a negotiator used by the provider to col- 
lect data, compute and negotiate resource consumption. RASp consists of a Manager With Negotiator 
{MWNp), Metering Service (MSp) and Accounting Service (ASp). The MSp collects data about resource 
consumption caused by all uploaded requests and stored them in a permanent tile. It records the follow- 
ing data user Id, request Id, request time stamp, request arrived time and number of bytes transferred per 
request. 

ASp determines the (SP, EP) for each C/,, obtains the metered data from MSp, calculates the average 
TT and produces a standard accounting record SR'I for each Clf . The ASp uses equation |5] to compute 
TT of a request: 

TT = Request ArrivalTime — RequestTimeStamp (5) 
The average TT is calculated by equation |6] 

TTave = ( V TT^ | * - (6) 

The MWNp obtains S/?f from its ASp and sends it to the provider NR middleware. Another re- 
sponsibility of of MWNp is to negotiate with the consumer RASc . In the negotiation steps, the MWNp 
receives negotiation requests with the consumer's accounting parameters and the negotiator counter from 
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Figure 6: Architecture and online dispute resolution protocol. 



the RASc- MWNp obtains the negotiator requests and sends it to ASp who compares them with its ac- 
counting parameters. The ASp obtains the parameter or parameters which cause the conflict, updates the 
negotiator counter and sends the negotiation response to the MWNp who sends it to the RASc and waits 
for new negotiation requests. 



6.2 Consumer Resource Accounting Service 

Similarly to the provider's resource accounting system, the RASc is a service with a negotiator that is used 
by the consumer to collect data, compute, negotiate and produce decisions about storage consumption 
for each CI. RASc consists of a Manager With Negotiator (MWNc), Metering Service (MSc), Accounting 
Service (ASc) and a Comparatorc (see Figure [6]). The MSc collects data about resource consumption 
caused by all uploaded requests and stores them in a permanent file. It records the following data request 
Id, request time stamp, number of bytes transferred per request. RASc works as follows: 

1. MWNc receives the provider's standard accounting record SR^^ from the consumer NR middleware. 

2. The MWNc sends the available accounting parameters /?■ to ASc 

3. ASc configures its accounting model using the details. consists of C/,- and TT]. 

4. ASc obtains metered data from MSc based on the C/, , computes SR^ using its accounting model 
and sends it the MWNc- 

5. MWNc sends SRI, SRf to the Comparatorc, who compares them and returns the decision decnt to 
the MWNc. 

If decni = No the MWNc starts negotiation with RASp aiming at solving the dispute. When MWNc 
receives a negotiation response it resets the value of R'j according to the evidence received from the 
negotiation response and executes steps 2 to 5. When the negotiation is completed or MWNc obtains 
decni = Yes from the Comparatorc, the MWNc sends the decni to NR middleware and waits for a new 
an accounting record from the consumer NR middleware. 
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6.3 P2P Protocol for Online Dispute Resolution 

The base of our dispute resolution protocol is the exchange of SP, EP of the interval under question and 
the average TT. The main idea of this protocol that the provider computes the resource consumption for 
each consumption interval C/, and sends it through its NR middleware to the consumer. The consumer 
sends the record to its RASc which compares it with its own record and in case of dispute it starts a 
negotiation with the RASp. The RASc sends a negotiation request to RASp. The negotiation request 
should contain the accounting parameters of the RASc- The RASp compares the consumer parameters 
with its parameters, finds the parameter or parameters that cause the conflict and sends them to the RASc 
with a negotiation response. The RASc uses the provider's parameters to compute storage consumption. 
It then compares the two records: if they match it sends a decni to the consumer NR middleware and 
waits for a new accounting record. Otherwise, it sends a new negotiation request until the end of the 
protocol. The negotiation protocol is completed either when the RASc has received a stop negotiation 
message from the RASp or the RASc obtains an agreed decision. 

We now show the psudocode of the protocol for dispute resolution. We use the following notation: 

# nc?= consumer negotiator 

# decni= decision 

# SRf= provider's storage consumption record containing storage consumption (SC) 

# 5/??= consumer's storage consumption record containing storage consumption (SC) 

# TT= transmission time 

# SP= start point, EP= end point 

# R'j= consumer's accounting parameters SP[, EPf, TTf 
#Rf= provider's accounting parameters SPf, EPf, TTf 

# NReqi= negotiation request i?^, nc? 

# NReSi= negotiation response /?f , ncf 



MWNj (SRf ) 

#nc. =true 
# decnpfalse 

// select the start the and end point of Clj 
Set the value of SP," = X, SP," =Y,TT,' =0 
// set the aecgunting paransters 

X = (( s^. Eif).Ti;'} 

// get the storage consumption from AS^ 

SR' = call AS„ ( R; ) 
While (Idecn, ) 
{ 

If ( SK^ gKT} t ttecn,=true and ilC,=fa|se} ffagre. decn 
else 

(If p iMf til Stort negotiation 

// set negotiation request 
NReqi = { R° , nc' } 
// send the consumer accounting parameters 
ewlene^ ts 

NRes, = Call RASp tltlReq,! 
// obtains the provider accounting parameters 
from NResi and update the consumer 
accounting parameters 

B,j a £F J/ update aeci {HMtteMt 
= 11^ A update the He. 
// re^compute storage consumption according 

to the provider parameters 

SR" = call AS, (R;' ) 
W/ e nd if 

else{ breiak ^Sdeeninet^gi^e and stop 
} // end while loop 

Send decn, to MyVN^ and stop // send decision 
and stop 
H/endMWN^ 



RASp (NReq, ) ^ 
{ // Qbtairis Consuftief Pm. ParSm^fia %^ F*®*^, NReqj 

limtnpafe the prov. arid the eons, stccbunting parameters 
lf(Ci; != Clf ) // compare CIs 



{ 



Rf = {( SP'.EP' ), 0} // Send Prov. CI 



} 

else if ( TT » != TT; ) // compare TT 
{ 

R ■ ={( SP • . EP," ), TT," } // send Prav. Gf and TT 
} else 
( 

ncf = false // no more evidences 

} 

Return NReSj = {(R^ ,ncf )} // return negotiation response 



as,(r:) 

{ 

// get a a list of meterinp data according to IheAcc. Parameteis from MSc( RT ] 
// Compute the storage consumption 

SC =2SCUF- J wtierei. nisthe1=^andthelastfileinthe meteringdata. 



ms„(r;) 
{ 



gel 8%"Ef^ »B( tif K mum mtmta0% wamsftwf M ) 

lltsa a llsi i« niBliitng dib of all upkad tliss durti^ Inlsrvil - Tr,; B>,- n,) 

Return ( a list of mrteitng data) 

1 
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The idea of bilateral measurement of resource consumption with online dispute resolution of potential 
conflicts was firstly suggested in |5|, however, no protocol to solve the problem was discussed; in this 
respect, our work can be regarded as a step forward in this research direction. Online Dispute Resolution 
(ODR) systems based on Utility Theory have been studied by some authors (see for example ifTOl ). 
A particularity of these ODR systems is that they attempt to reach interest-based voluntary settlement 
agreements based on parties' preferences and tradeoff. In contrast, in our work disputes are solved — 
if possible — on the basis of evidences presented by the two parties involved in the conflict. Another 
particularity of these ODR is that they normally rely on a third party (e.g. arbitration) to help solve the 
dispute; we depart from this idea and suggest a peer-to-peer conflict resolution protocol. 

The use of middleware interceptors to monitor Service Level Agreements-regulated interactions be- 
tween two parties is discussed in |61. 

The problem of deciding the physical location of the components of metering services to measure 
resource consumption is related to the concept of monitorability discussed in |8|. In accordance with 
these authors, a given service level parameter (for example, response time, storage consumption, etc.) 
is unmonitorable, monitorable by a trusted third party, monitorable by one party, monitorable by both 
parties. In |5| the authors explain that the monitorability of a given parameter depends on several factors; 
among the most important ones are the accuracy of the metering, the physical location of the application 
(one or many) that affects the parameter, the physical location of the metering service and the trust 
assumptions about the metering service and its location. 

In m the authors discuss a protocol to provide non-repudiable evidence of services consumption 
in mobile internet services. Non-repudible evidence is used by the provider to prove the correctness of 
his bill and allows the consumer to verify the service consumption. The protocol suffers from several 
limitations. For example, it involves an online trusted third party to insure fairness of the protocol. 
Likewise, it works for time-base accounting only. Furthermore, the non-repudiable service increases 
the number of messages exchanged through the network. Consequently, the additional messages reduce 
the effective bandwidth for users' data traffic. Moreover, if disputes over a service interval appear, the 
service is simply terminated; when this happens, the provider looses money because the consumer would 
have already consumed part of the service interval. Our work constrasts with this approach in that we try 
to solve conflict when they apper rather than terminating the service. 



8 Conclusion and Future Work 



In bilateral accounting of resource consumption the consumer and the provider independently measure 
resource consumption, compare their outcomes and try to agree on a a single outcome. In this paper 
we have discussed when, why and where conflicts might happen in bilateral accounting for storage 
consumption. We propose a peer-to-peer online protocol to be executed between the consumer and 
provider to solve conflics over the consumer's consumption. In future, we are planning to study other 
parameters (different techniques to collect data about resource consumption) that might cause disputes 
over storage consumption. 
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